Mesa regression testing 


Problems / Goals 


Thousanas of different game vvorkloads on Steam 
Rendering and perf regressions are frequent 
Unit tests don't catch everything 


Manual testing is good, but doesn't help with the long tail of popular games 


LunarG testing lab 


Traces of many Steam games in a database 

Apitrace for OpenGL, vktrace for Vulkan 

Finds rendering differences and perf differences 

Testing of tagged releases, RCs 

Slower turnaround than unit tests, master testing when time allows 


Pre-merge testing 


DEMO 


https://share.lunarg.com 


https://share.lunarg.com/test/report/202#overview 


What happens after testing” 


Some human work reguired to turn testing data into useful bug reports 
Solution: make results public easy to browse and presented efficiently 
Will make game license and trace available to devs if asked 
Community/companies can monitor the permutations they care about 


Very early stages, looking for feedback 


